DESCRIPTION OF CHANGES:

The latest versions of MET/METplus have introduced a new tool named GenEnsProd that performs many of the ensemble-related calculations that EnsembleStat used to do. Although EnsembleStat can still perform these calculations, that will change in future versions of MET/METplus. Thus, this PR creates new vx tasks that call GenEnsProd to perform the necessary calculations, removing them from the EnsembleStat tasks. Details:

• Add new METplus config files for GenEnsProd.
• Modify exregional_run_met_genensprod_or_ensemblestat.sh to enable it to call GenEnsProd (as well as EnsembleStat).
• Modify the EnsembleStat config files so that EnsembleStat no longer calculates the ensemble statistics that are now the purview of GenEnsProd.
• Modify the ex-scripts for the ensemble mean and probabilistic tasks so that they look for forecast input from the output of the GenEnsProd tasks, not the output of the EnsembleStat tasks.
• Modify all the METplus config files so that they are as similar to each other as possible (for possible consolidation later on).
• For clarity and correctness, rename the workflow configuration files verify.yaml and verify_ensgrid.yaml to verify_det.yaml and verify_ens.yaml, respectively.
• Add new yaml configuration file named verify_pre.yaml that contains (meta)tasks that are prerequisites for both deterministic and ensemble vx (these tasks were previously included in verify_det.yaml).
• Modify the default ensemble vx workflow configuration file verify_ens.yaml to include the new GenEnsProd tasks.
• Modify verify_det.yaml and verify_ens.yaml so that vx task dependencies can be formulated more precisely, e.g. have dependencies on member-specific tasks instead of whole metatasks. This includes splitting metatasks into two or more other ones, moving tasks from one metatask to another, etc.
• Modify the WE2E configuration files for the four vx tests (MET_verification, MET_verification_only_vx, MET_ensemble_verification, and MET_ensemble_verification_only_vx) to reflect the changes in the default vx workflow configuration files verify_pre.yaml, verify_det.yaml, and verify_ens.yaml.
• Introduce the new experiment variable VX_FIELDS to enable the user to specify the fields or groups of fields on which to perform verification.
• Introduce the new experiment variable VX_APCP_ACCUMS_HH to enable the user to specify the accumulation hours for which to perform verification on accumulated precip (APCP).
• Modify setup.py to remove any [meta]tasks for which fields are not specified in VX_FIELDS.
• Modify setup.py so that for configuration parameters that are arrays/lists, each value of the list is checked to ensure it is valid (if that variable has a corresponding valid_vals_ array in valid_param_vals.yaml).
• Make modifications to vx ex-scripts as well as config_defaults.sh to ensure that the vx tests succeed in NCO mode.
• Make sure that the experiment variable ENS_TIME_LAG_HRS is set and accessed correctly for both deterministic and ensemble configurations. For a deterministic configuration, this is a 1-element array containing the time lag (if any, in units of hours), while for an ensemble configuration it contains NUM_ENS_MEMBERS elements containing the time lag for each element.
• Add a new WE2E test (MET_ensemble_verification_only_vx_time_lag) that performs deterministic and ensemble verification on a 2-member ensemble with one time-lagged and one non-time-lagged member.
• Add the existing WE2E test MET_ensemble_verification_only_vx and the new test MET_ensemble_verification_only_vx_time_lag to the list of comprehensive WE2E tests.
• Include hour 0 in the verification of MRMS and NDAS fields.
• Fix bugs in exregional_get_obs_[ccpa|mrms|ndas].sh ex-scripts.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

TESTS CONDUCTED:

• hera.intel
• orion.intel
• cheyenne.intel
• cheyenne.gnu
• gaea.intel
• jet.intel
• wcoss2.intel
• NOAA Cloud (indicate which platform)
• Jenkins
• fundamental test suite
• comprehensive tests (specify which if a subset was used)

I ran the following vx WE2E tests on Hera with intel, both in community mode and in NCO mode:

MET_verification
MET_verification_only_vx
MET_ensemble_verification
MET_ensemble_verification_only_vx
MET_ensemble_verification_only_vx_time_lag

I also ran the following fundamental suite of tests:

grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_RAP_suite_RRFS_v1beta
nco_grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_timeoffset_suite_GFS_v16
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v15p2
grid_RRFS_CONUS_25km_ics_FV3GFS_lbcs_FV3GFS_suite_GFS_v17_p8_plot
grid_RRFS_CONUScompact_25km_ics_HRRR_lbcs_HRRR_suite_HRRR
grid_SUBCONUS_Ind_3km_ics_HRRR_lbcs_RAP_suite_WoFS_v0
grid_RRFS_CONUS_25km_ics_NAM_lbcs_NAM_suite_GFS_v16

All tests were successful.

DOCUMENTATION:

Documentation is needed but will be added once the whole set of vx PRs has been merged (see Issue #630).

CHECKLIST

• My code follows the style guidelines in the Contributor's Guide
• I have performed a self-review of my own code using the Code Reviewer's Guide
• I have commented my code, particularly in hard-to-understand areas
• My changes need updates to the documentation. I have made corresponding changes to the documentation
• My changes do not require updates to the documentation (explain).
• My changes generate no new warnings
• New and existing tests pass with my changes
• Any dependent changes have been merged and published